Auditory Stream Segregation in Auditory Scene Analysis with a Multi-Agent System

We propose a novel approach to auditory stream segregation which extracts individual sounds (auditory stream) from a mixture of sounds in auditory scene analysis. The HBSS (Harmonic-Based Stream Segregation) system is designed and developed by employing a multi-agent system. HBSS uses only harmonics as a clue to segregation and extracts auditory streams incrementally. When the tracer-generator agent detects a new sound, it spawns a tracer agent, which extracts an auditory stream by tracing its harmonic structure. The tracer sends a feedforward signal so that the generator and other tracers should not work on the same stream that is being traced. The quality of segregation may be poor due to redundant and ghost tracers. HBSS copes with this problem by introducing monitor agents, which detect and eliminate redundant and ghost tracers. HBSS can segregate two streams from a mixture of man’s and woman’s speech. It is easy to resynthesize speech or sounds from the corresponding streams. Additionally, HBSS can be easily extended by adding agents of a new capability. HBSS can be considered as the first step to computational auditory scene analysis. Introduction Over the past years a considerable number of studies have been made on human auditory mechanisms. Although we have many techniques for processing particular sounds such as speech, music, instruments, and the sounds made by specific devices, we don’t have enough mechanisms for processing and understanding sounds in real acoustic environments. Research into the latter is being made in the field of Auditory Scene Analysis (Bregman 1990)) which is to speech recognition is what scene analysis is to character recognition. Auditory scene analysis is a difficult challenging area, partly because acoustic theory is not still rather inadequate (e.g., there is no good acoustic design methodology for concert halls), and partly because most research in acoustics has been focused exclusively on speech and music, ignoring many other sounds. Additionally, the reductionist approach to auditory scene 100 The Arts analysis, which tries to sum up various techniques for handling individual sounds, is not promising. Looking and listening are more active than seeing and hearing (Handel 1989). The essentials of our approach to auditory scene analysis are twofold: e Active perception of observer looking and listening rather than seeing and hearing, and e Multi-sensor perception may use multi-modal information perceived by means of sensor organs The multi-agent system was recently proposed as a new modeling technology in artificial intelligence (Brooks 1986) (M aes 1991) (Minsky 1986) (Okuno 1993). We assume like Minsky that an agent has a limited capability, although in Distributed Artificial Intelligence, an agent is supposed to be much more powerful like a human being than ours. Each agent has its own goal and competes and/or cooperates with other agents. Through interactions among agents, intelligent behavior emerges (Okuno & Okada 1992). Consider the approach that the multi-agent paradigm is applied to model auditory scene analysis. We expect that it will enhance the following functionalities: (1) Goal-Orientation Each agent may have its own goal. (2) Adaptability According to the current situation, the behavior of the system varies between reactive and deliberate. (3) Robustness The system should respond sensibly even if the input contains errors, or is ambiguous and incomplete. (4) Openness The system can be extended by adding agents of new capabilities. It can also be integrated into other systems as a building block. In this paper, auditory stream segregation, the first stage of auditory scene analysis, is modeled and implemented by a multi-agent system. The rest of this paper is organized as follows: Section 2 investigates issues in auditory stream segregation. In Section 3, the basic system of auditory stream segregation with a multiagent system is explained and evaluated to identify its problems. Section 4 presents and evaluates the HBSS (Harmonic-Based Stream Segregation) that copes with the problems. Related work and the conclusions are given in Section 5 and 6, respectively. From: AAAI-94 Proceedings. Copyright © 1994, AAAI (www.aaai.org). All rights reserved. Auditory stream fop auditory scene analysis